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BACKGROUND OF THE INVENTION 



This invention relates to an image processing 
apparatus and method for processing an image signal 
10 input from an image sensing device or the like, and to a 
computer-readable storage medium used in the apparatus 
and method. 

Recent progress in digital signal processing 
technologies has led to major developments in the video 

15 field. In image sensing devices such as digital 

cameras, signal processing circuitry has been digitized 
to make possible devices exhibiting no signal 
deterioration or aging, unlike the case with devices 
composed of conventional analog circuits. 

20 On the other hand, schemes in which signal 

processing is implemented not by hardware based upon 
digital circuits but by software using a CPU have also 
been proposed. Because the content of signal processing 
is decided by a program built into a ROM, such schemes 

25 make possible adaptive processing in which the content 
of processing is selected in accordance with the state 
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of the input image. 

With the above-described signal processing approach 
that relies upon digital circuitry, hardware 
corresponding to all selection candidates must be 
5 provided in order to execute adaptive processing. The 
problem that arises is a great increase in the scale of 
the circuitry. 

With the approach that relies upon software for 
processing, on the other hand, processing speed is 
10 slower than with the hardware-implemented approach and 

ordinary moving-picture signals (e.g., 720 x 240 pixels; 

60 fields per second) cannot be processed in real time. 

The above-mentioned problems encountered with both 
approaches arise particularly when trying to reduce a 
15 decline in image quality due to saturation of the image 
sensor caused by a subject having a high degree of 
luminance . 

SUMMARY OF THE INVENTION 

20 

Accordingly, an object of the present invention is 
to execute high-quality image processing by a simple 
circuit arrangement . 

According to the present invention, the foregoing 
25 object is attained by providing an image processing 

apparatus comprising: detecting means for detecting, in 



- 2 - 



an entered image signal, a high- luminance portion that 
exceeds a predetermined value; generating means for 
generating a control signal, which has a prescribed 
waveform at the periphery of the high- luminance portion 
5 of the image signal, in dependence upon the detection 
made by the detecting means; separating means for 
separating a color signal from the image signal ; and 
suppression means for suppressing the separated color 
O signal by the control signal. 

Ul 10 Further, according to the present invention, the 

M foregoing object is attained by providing an image 

M processing method comprising: a detecting step of 

detecting, in an entered image signal, a high- luminance 
j|j portion that exceeds a predetermined value; a generating 

"5 15 step of generating a control signal, which has a 

:;:? prescribed waveform at the periphery of the sensed high- 

luminance portion of the image signal; a separating step 
of separating a color signal from the image signal; and 
a suppression step of suppressing the separated color 
20 signal by the control signal. 

Further, according to the present invention, the 
foregoing object is attained by providing a computer- 
readable storage medium storing a program for executing: 
detection processing for detecting, in an entered image 
25 signal, a high- luminance portion that exceeds a 
predetermined value; generation processing for 
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generating a control signal, which has a prescribed 
waveform at the periphery of the sensed high- luminance 
portion of the image signal; separation processing for 
separating a color signal from the image signal; and 
5 suppression processing for suppressing the separated 
color signal by the control signal. 

Other objects and advantages besides those 
discussed above shall be apparent to those skilled in 
^ the art from the specification of a preferred embodiment 

10 of the invention which follows. In the description, 

reference is made to accompanying drawings, which form a 
part thereof, and which illustrate an example of the 
invention. Such example, however, is not exhaustive of 
the various embodiments of the invention, and therefore 
15 reference is made to the claims which follow the 

description for determining the scope of the invention. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



20 Fig. 1 is a block diagram illustrating an image 

sensing device which includes an image processor in 
accordance with an embodiment of the present invention; 

Figs. 2A and 2B are diagrams showing an arrangement 
that is useful in describing the principles of the 
25 present invention; 

Fig. 3 is a block diagram showing the principal 
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components in Fig. 1; and 

Fig. 4 is a flowchart illustrating processing 
executed by a pattern controller in Fig. 1. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 

A preferred embodiment of the present invention 
will now be described with reference to the drawings, 
^j^^/^ig. 1 is a block diagram illustrating an image 
10 sensing\drive which includes an image processor 

according to this embodiment of the present invention. 

As shown in Fig. 1, the image sensing device 
includes a taking lens 1; a CCD 2, which is a color 
image sensor; a sample-and-hold unit 3 for rendering the 
15 output signal of the CCD 2 continuous; an A/D converter 
.4; a color separator 5 for separating a luminance signal 
Y and color signals R, G, B from a digital image signal 
obtained by the A/D converter 4; a Y process unit 6 for 
subjecting the luminance signal Y to processing such as 
20 a gamma correction, black-level clipping, white-level 
clipping and contour emphasis; a field memory 7 for 
delaying the input luminance signal by one field; and an 
output terminal 8 for the luminance signal Y. 

The image sensing device further includes an RGB 
25 process unit 9 for subjecting the color signals R, G, B 
to processing such as white balance control, a gamma 
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correction, black-level clipping and white-level 
clipping; a color-difference matrix unit 10 for 
subjecting the R, G, B signals to matrix processing, 
thereby to generate color difference signals, and for 
5 multiplexing the color difference signals; a field 

memory 11 for delaying the multiplexed color 1 difference 
signal by one field; a suppression unit 12 for 
suppressing the multiplexed color difference signal in 
y conformity with a suppression signal Ss, described 

Ji^ 10 later; and an output terminal 13 for a multiplexed color 

H difference signal C. 

I'* The image sensing device further includes a 

35 saturation detector 14 for detecting saturation of the 

|y digital image signal; a first-in first-out (FIFO) memory 

J'fi 15 15; a control signal generator 16 for generating a 

j!S control signal; a memory 17 for storing the control 

signal; and a suppression-signal generator 18 for 
reading the control signal out of the memory 17, 
generating the suppression signal Ss and outputting the 
2 0 same. 

The operation of this arrangement will now be 
described. 

The image of a subject (not shown) is formed on the 
photoelectric converting surface of the CCD 2 by the 
25 taking lens 1. This image is composed of electric 
charge obtained by photoelectric conversion. The 
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electric charge within the CCD 2 is transferred 
successively by driving pulses generated by a driving- 
pulse generating circuit (not shown) , is converted to 
voltage at the output of the CCD 2 and is output as a 
5 captured- image signal. The captured- image signal is 
sampled continuously by the sample-and-hold unit 3 and 
is converted to a digital image signal by the A/D 
converter 4 . 

O The digital image signal is converted to the 

in 10 luminance signal Y and color signals R, G # B by the 

H color separator 5. The luminance signal Y is subjected 

f 3 

to processing such as a gamma correction, black- level 

-.. i 

S : clipping, white-level clipping and contour emphasis in 

m the Y process unit 6 and then is delayed by one field by 

^5 15 the field memory 7 . The resulting signal is output to 

equipment such as a television monitor or VTR, along 
with a C signal, described later, from the Y output 
terminal 8 . 

The color signals R, G, B in the output of the 
20 color separator 5 are subjected to processing such as 
white balance control, a gamma correction, black-level 
clipping and white- level clipping in the RGB process 
unit 9 and then are converted to color difference 
signals R-Y and B-Y in the color-difference matrix unit 
25 10, after which the signals are multiplexed by dot- 
sequencing to provide a multiplexed color difference 
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signal. The multiplexed color difference signal is 
delayed by one field in the field memory 11 and is 
suppressed in conformity with the suppression signal Ss 
in the suppression unit 12. The resulting signal is 
5 output from the C output terminal 13 . 

The digital image signal output of the A/D 
converter 4 has its saturation detected by the 
saturation detector 14 through a method such as one that 
$ > : £ detects a portion of the signal that exceeds a 

™J 10 predetermined threshold value. A signal indicative of 

H the detected saturation is stored temporarily in the 

O 

FIFO memory 15. The signal indicative of saturation 
a stored in the FIFO memory 15 is converted by the control 

jfy signal generator 16 to a control signal for suppressing 

^5 15 the color signals. The control signal is then written 

S to the memory 17 . The control signal that has been 

written to the memory 17 is read out by the suppression- 
signal generator 18 and is output as the suppression 
signal Ss, namely the control signal of the suppression 
20 unit 12. This control signal suppresses the above- 
mentioned color signals. 

Fig. 2A and 2B are diagrams useful in describing 
the principles of the embodiment. 

Fig. 2A illustrates a decline in image quality 
25 caused by a high- luminance subject. By way of the 

example, the portion illustrated as the high- luminance 
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portion is a subject which causes sunlight to impinge 
upon the taking lens 1 by reflection at a mirror 
surface. Because the high- luminance portion generates 
electric charge in the CCD 2 that exceeds the saturation 
5 level, the color carrier signal in the output signal 
vanishes and this causes an image deterioration 
phenomenon referred to as high- luminance false color. 
This makes it necessary to suppress the color signals. 
Phenomena that occur at the portion indicated in 

10 the area of image deterioration surrounding the high- 
luminance portion include aberration of the taking lens 
1, particularly aberration referred to as axial 
chromatic aberration, in which focal length changes 
owing to the wavelength of the light; optical 

15 deterioration such as flare and ghosts; and so-called 

blooming, in which electric charge produced on the CCD 2 
overflows at the periphery. As a consequence, 
unnecessary coloration occurs and results in a decline 
in image quality. 

20 Fig. 2B illustrates a color-signal suppression 

characteristic with regard to the area of image 
deterioration. As mentioned above, unnecessary 
coloration is produced in the high- luminance portion and 
at the periphery of the high- luminance portion and, as a 

25 consequence, it is necessary to suppress the color 

signals. The characteristic of this phenomenon is as 
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follows : 

(1) The unnecessary colors are centered on the 
high- luminance portion and decrease as distance from the 
center of the high- luminance portion increases. 
5 (2) If there is a discontinuous characteristic 

between the portion subjected to suppression and the 
periphery thereof, a dummy contour will be produced and 
image quality will undergo a marked decline. 
O For these reasons, it is necessary to adopt a 

IB 10 suppression characteristic in which color gain is made 

!«* zero in the high- luminance portion and suppression is 

H reduced as the periphery is approached and is eliminated 

5 : at a location beyond a predetermined distance from the 

iy high- luminance portion. Fig. 2B illustrates this 

5 :=s = 

15 characteristic . 
:;^f Fig. 3 is a block diagram illustrating the details 

of construction of the components 14 to 18, which are 
the principal components in Fig. 1. 

As shown in Fig. 3, the saturation detector 14 

20 includes a register 101 for storing a predetermined 
threshold value and a comparator 102. The control 
signal generator 16 includes a pattern controller 103; a 
readout register 104 for storing data that has been read 
out of the FIFO 15; an address generator 105 for 

25 generating a write address of the memory 17; adders 106 
and 107; and a pattern generator 108 for generating a 
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predetermined pattern. The suppression-signal generator 
18 includes a read-out address generator 109 for 
generating a read-out address of the memory 17, and a 
level converter 110. 
5 The operation of the components described above 

will now be described. 

A digital image signal SAD, which is the output of 
the A/D converter 4, is compared in the comparator 102 
with a predetermined threshold value from the register 

10 101. The output Sw of the comparator 102, which output 
corresponds to the saturated portion of the digital 
image signal, is stored in the FIFO memory 15 as a 
horizontal scanning position PH and vertical scanning 
position PV generated by a timing generator, which is 

15 not shown. 

When a scanning position corresponding to the 
saturated portion is stored in the FIFO memory 15, as 
mentioned above, a FIFO-empty signal SFE, which 
represents whether information has been stored in the 

20 FIFO memory 15, is read out by the pattern controller 
103 . When the signal SFE indicates that the FIFO is 
empty, no operation is performed. 

When the signal SFE indicates that the FIFO is not 
empty, the pattern controller 103 first controls the 

25 readout register 104 to read out data SRD, which 

represents the scanning position that corresponds to the 
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saturated portion and that is stored in the FIFO memory 
15, and to load this data in the readout register 104. 

Next, the pattern controller 103 controls the 
address generator 105 so that the latter generates a 
5 series of horizontal and vertical addresses AHS, AVS in 
a predetermined order. The generated horizontal and 
vertical addresses AHS, AVS are added by the adders 106, 
107, respectively, to the horizontal and vertical 
s -=3 positions, respectively, stored in the readout register 

10 104, whereby a conversion is made to addresses the 

H center of which is the saturated portion. The 

O 

m conversion outputs are input to the memory 17 as 

5, horizontal and vertical addresses AH, AV, respectively. 

iy The pattern controller 103 further controls the 

15 pattern generator 108 so that the latter generates a 
J-S prescribed two-dimensional waveform that corresponds to 

the addresses AHS, AVS generated by the address 
generator 105. At this time a value DR at the 
horizontal and vertical addresses of the memory 17 is 
20 read out, this value is compared with the value of the 

waveform generated in the manner described above and, by 
way of example, a larger value is written to the memory 
17 as write data DW at a location designated by the 
horizontal and vertical addresses AHS, AVS. 
25 Thus, the operation described above is such that if 

there are a plurality of saturated areas, a suppression 
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signal that gives priority to the largest degree of 
suppression can be generated. 

When the generation of the series of addresses and 
of the two-dimensional waveform is completed, the 
5 pattern controller 103 reads out the FIFO-empty signal 
SFE again and repeats the above-described operation 
until the FIFO is emptied. 

The two-dimensional waveform thus written to the 
O memory 17 is then read out. Specifically, the read-out 

ill 10 address of the memory 17 is generated by the read-out 

Q 

M= address generator 109 in dependence upon synchronizing 

1^ signals HD and VD generated by a synchronizing signal 

« generator (not shown) , and the two-dimensional waveform 

ry is read out based upon this read-out address in 

15 accordance with television scanning. The data that has 
:;* r been read out is converted by the level converter 110 to 

a level suited to the suppression signal, the resulting 
signal is applied to the suppression unit 12 as the 
suppression signal Ss, and the color signals in the 
20 saturated portion of the CCD and in the peripheral 

portion thereof are suppressed in the manner described 
above . 

Fig. 4 is a flowchart illustrating processing in a 
case where the pattern controller 103 is implemented by 
25 a microcomputer. 

After processing starts at step S201 in Fig. 4, 
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the FIFO-empty signal SFE that indicates whether 
information has been stored in the FIFO memory 15 is 
written in at step S202. This is followed by step S203, 
at which it is determined whether the FIFO-empty signal 
5 SFE indicates that the FIFO is empty. Control returns 
to S2 02 if the FIFO is empty. 

If the FIFO is not empty, then control proceeds to 
S2 04, at which the data SRD representing the scanning 
position stored in the FIFO memory 15 is read out to the 

10 readout register 104. Next, at S2 05, the address 

generator 105 is controlled so as to generate the series 
of horizontal and vertical addresses of the 
predetermined order. Then, at S2 06, the pattern 
generator 108 is controlled so as to generate the 

15 predetermined two-dimensional waveform that corresponds 
to the addresses generated by the address generator 105. 
This is followed by S207, at which it is determined 
whether the address generator 105 has finished address 
generation. Control returns to S2 07 if the answer is 

20 "NO" and to S202 if the answer is "YES" , after which the 
above-described control is repeated. 

A storage medium in accordance with the present 
invention will now be described. 

Though the embodiment illustrated in Figs. 1 and 3 

2 5 is implemented by hardware, it can also be implemented 
by a computer system having a CPU and memory. In case 
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of implementation by a computer system, the memory 
constitutes a storage medium in accordance with the 
present invention. A program for executing the 
processing according to the flowchart of Fig. 4 and the 
5 operation described in this embodiment is stored in this 
storage medium. 

This storage medium may be a semiconductor memory 
such as a ROM or RAM, an optical disk, a magneto-optic 
disk or a magnetic storage medium, etc., and these may 
10 be constructed in the form of a CD-ROM, floppy disk, 
magnetic card, magnetic tape or non-volatile memory 
card, etc. 

Accordingly, functions equivalent to those of the 
foregoing embodiment can be implemented, similar effects 

15 can be obtained and the object of the present invention 
can be attained by using the storage medium in a system 
or apparatus other than that according to Figs . 1 and 4 , 
and reading out and executing program code , which has 
been stored on the storage medium, by this system or by 

20 a computer. 

Further, functions equivalent to those of the 
foregoing embodiment can be implemented, similar effects 
can be obtained and the object of the invention can be 
attained in a case where an operating system or the like 

25 running on a computer performs all or a part of the 

processing or in a case where, after the program codes 
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read from the storage medium are written in a function 
expansion board inserted into the computer or in a 
memory provided in a function expansion unit connected 
to the computer, a CPU or the like contained in the 
5 function expansion board or function expansion unit 
performs all or a part of the entire process in 
accordance with the designation of program codes. 

In accordance with the present invention, as 
described above, it is possible to reduce a decline in 

10 the image quality of a high- luminance portion of an 
image signal. 

Further, since information regarding a saturated 
area is stored in memory in order to obtain the above- 
described effects, means for generating a control signal 

15 can be realized by a circuit having a small processing 
capability because it is no longer always necessary to 
execute processing in real time. This makes it possible 
to obtain the best characteristic at low cost. 
Further, it is possible to reduce image 

20 deterioration of an area caused by light having an 

intensity greater than the saturation level of the image 
sensor on which the light impinges. 

In addition, color-signal suppression can be 
carried out continuously and smoothly, thereby making it 

25 possible to eliminate unnecessary coloration of an image 
signal as well as dummy contours. 
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The present invention is not limited to the above 
embodiments and various changes and modifications can be 
made within the spirit and scope of the present 
invention. Therefore, to apprise the public of the 
scope of the present invention, the following claims are 
made . 
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